ABSTRACT
Purposes: Artificial Intelligence (AI) models are constantly developing to help clinicians in challenging tasks such as classification of images in radiological practice. The aim of this work was to compare the diagnostic performance of an AI classifier model developed in our hospital with the results obtained from the radiologists reading the CT images in discriminating different types of viral pneumonia. Material(s) and Method(s): Chest CT images of 1028 patients with positive swab for SARS-CoV-2 (n=646) and other respiratory viruses (n=382) were segmented automatically for lung extraction and Radiomic Features (RF) of first (n=18) and second (n=120) order were extracted using PyRadiomics tools. RF, together with patient age and sex, were used to develop a Multi-Layer Perceptron classifier to discriminate images of patients with COVID-19 and non-COVID-19 viral pneumonia. The model was trained with 808 CT images performing a LASSO regression (Least Absolute Shrinkage and Selection Operator), a hyper-parameter tuning and a final 4-fold cross validation. The remaining 220 CT images (n=151 COVID-19, n=69 non-COVID-19) were used as independent validation (IV) dataset. Four readers (three radiologists with >10 years of experience and one radiology resident with 3 years of experience) were recruited to blindly evaluate the IV dataset using the 5-points scale CO-RADS score. CT images with CO-RADS >=3 were considered "COVID-19". The same images were classified as "COVID-19" or "non-COVID-19" by applying the AI model with a threshold on the predicted values of 0.5. Diagnostic accuracy, specificity, sensibility and F1 score were calculated for human readers and AI model. Result(s): The AI model was trained using 24 relevant features while the Area under ROC curve values after 4-fold cross validation and its application to the IV dataset were, respectively, 0.89 and 0.85. Interreader agreement in assigning CO-RADS class, analyzed with Fleiss' kappa with ordinal weighting, was good (k=0.68;IC95% 0.63-0.72) and diagnostic performance were then averaged among readers. Diagnostic accuracy, specificity, sensibility and F1 score resulted 78.6%, 78.3%, 78.8% and 78.5% for AI model and 77.7%, 65.6%, 83.3% and 72.0% for human readers. The difference between specificity and sensitivity observed in human readers could be related to the higher rate of false positive due to the higher incidence of COVID-19 patients in comparison with other types of viral pneumonitis during the last 2 years. Conclusion(s): A model based on RF and artificial intelligence provides comparable results with human readers in terms of diagnostic performance in a classification task.Copyright © 2023 Southern Society for Clinical Investigation.